Introduction to Neural Networks Project¶
Part A¶
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
import tensorflow
from keras import optimizers
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.regularizers import l2
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import ReduceLROnPlateau, EarlyStopping
from tensorflow.keras.models import load_model
from tensorflow.keras.losses import SparseCategoricalCrossentropy
1.A Read the ‘Signals.csv’ as DatFrame and import required libraries.¶
signalsDf = pd.read_csv("Signal.csv")
signalsDf.head
<bound method NDFrame.head of Parameter 1 Parameter 2 Parameter 3 Parameter 4 Parameter 5 \
0 7.4 0.700 0.00 1.9 0.076
1 7.8 0.880 0.00 2.6 0.098
2 7.8 0.760 0.04 2.3 0.092
3 11.2 0.280 0.56 1.9 0.075
4 7.4 0.700 0.00 1.9 0.076
... ... ... ... ... ...
1594 6.2 0.600 0.08 2.0 0.090
1595 5.9 0.550 0.10 2.2 0.062
1596 6.3 0.510 0.13 2.3 0.076
1597 5.9 0.645 0.12 2.0 0.075
1598 6.0 0.310 0.47 3.6 0.067
Parameter 6 Parameter 7 Parameter 8 Parameter 9 Parameter 10 \
0 11.0 34.0 0.99780 3.51 0.56
1 25.0 67.0 0.99680 3.20 0.68
2 15.0 54.0 0.99700 3.26 0.65
3 17.0 60.0 0.99800 3.16 0.58
4 11.0 34.0 0.99780 3.51 0.56
... ... ... ... ... ...
1594 32.0 44.0 0.99490 3.45 0.58
1595 39.0 51.0 0.99512 3.52 0.76
1596 29.0 40.0 0.99574 3.42 0.75
1597 32.0 44.0 0.99547 3.57 0.71
1598 18.0 42.0 0.99549 3.39 0.66
Parameter 11 Signal_Strength
0 9.4 5
1 9.8 5
2 9.8 5
3 9.8 6
4 9.4 5
... ... ...
1594 10.5 5
1595 11.2 6
1596 11.0 6
1597 10.2 5
1598 11.0 6
[1599 rows x 12 columns]>
signalsDf.shape
(1599, 12)
signalsDf.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 1599 entries, 0 to 1598 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Parameter 1 1599 non-null float64 1 Parameter 2 1599 non-null float64 2 Parameter 3 1599 non-null float64 3 Parameter 4 1599 non-null float64 4 Parameter 5 1599 non-null float64 5 Parameter 6 1599 non-null float64 6 Parameter 7 1599 non-null float64 7 Parameter 8 1599 non-null float64 8 Parameter 9 1599 non-null float64 9 Parameter 10 1599 non-null float64 10 Parameter 11 1599 non-null float64 11 Signal_Strength 1599 non-null int64 dtypes: float64(11), int64(1) memory usage: 150.0 KB
signalsDf.describe()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 8 | Parameter 9 | Parameter 10 | Parameter 11 | Signal_Strength | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 | 1599.000000 |
| mean | 8.319637 | 0.527821 | 0.270976 | 2.538806 | 0.087467 | 15.874922 | 46.467792 | 0.996747 | 3.311113 | 0.658149 | 10.422983 | 5.636023 |
| std | 1.741096 | 0.179060 | 0.194801 | 1.409928 | 0.047065 | 10.460157 | 32.895324 | 0.001887 | 0.154386 | 0.169507 | 1.065668 | 0.807569 |
| min | 4.600000 | 0.120000 | 0.000000 | 0.900000 | 0.012000 | 1.000000 | 6.000000 | 0.990070 | 2.740000 | 0.330000 | 8.400000 | 3.000000 |
| 25% | 7.100000 | 0.390000 | 0.090000 | 1.900000 | 0.070000 | 7.000000 | 22.000000 | 0.995600 | 3.210000 | 0.550000 | 9.500000 | 5.000000 |
| 50% | 7.900000 | 0.520000 | 0.260000 | 2.200000 | 0.079000 | 14.000000 | 38.000000 | 0.996750 | 3.310000 | 0.620000 | 10.200000 | 6.000000 |
| 75% | 9.200000 | 0.640000 | 0.420000 | 2.600000 | 0.090000 | 21.000000 | 62.000000 | 0.997835 | 3.400000 | 0.730000 | 11.100000 | 6.000000 |
| max | 15.900000 | 1.580000 | 1.000000 | 15.500000 | 0.611000 | 72.000000 | 289.000000 | 1.003690 | 4.010000 | 2.000000 | 14.900000 | 8.000000 |
1.B Check for missing values and print percentage for each attribute.¶
signalsDf.isna().sum()/len(signalsDf)
Parameter 1 0.0 Parameter 2 0.0 Parameter 3 0.0 Parameter 4 0.0 Parameter 5 0.0 Parameter 6 0.0 Parameter 7 0.0 Parameter 8 0.0 Parameter 9 0.0 Parameter 10 0.0 Parameter 11 0.0 Signal_Strength 0.0 dtype: float64
signalsDf = signalsDf.dropna()
1.C Check for presence of duplicate records in the dataset and impute with appropriate method.¶
# Check number of duplicate records
signalsDf.duplicated().sum()
240
There are 240 dupicate values found.
signals_df = signalsDf.copy()
signals_df.drop_duplicates(keep = 'first', inplace = True)
signals_df.shape
(1359, 12)
1.D Visualise distribution of the target variable.¶
sns.histplot(data = signals_df, x = 'Signal_Strength');
plt.show()
sns.boxplot(data = signals_df, x = 'Signal_Strength');
plt.show()
sns.pairplot(data = signals_df);
plt.show()
1.E Share insights from the initial data analysis (at least 2).¶
Insights¶
- There are 1599 rows and 12 columns in the data frame.
- All values are numeric only.
- 240 records found duplicate, cleaned those rows. Leftare 1359 rows in the data frame.
- The target column Signal Strength contains integer values between 3 to 8.
- Most of the Signal_Strength are 5 or 6.
2.A Split the data into X & Y.¶
signals_X = signals_df.drop('Signal_Strength', axis = 1)
signals_Y = signals_df['Signal_Strength']
2.B Split the data into train & test with 70:30 proportion.¶
X_train, X_test, Y_train, Y_test = train_test_split(signals_X, signals_Y, test_size = 0.30, stratify = signals_Y, random_state = 1)
2.C Print shape of all the 4 variables and verify if train and test data is in sync.¶
print('X train:', X_train.shape, '| X test:', X_test.shape, '| Y train:', Y_train.shape, '| Y test:', Y_test.shape)
X train: (951, 11) | X test: (408, 11) | Y train: (951,) | Y test: (408,)
Y_train.value_counts()
Signal_Strength 5 404 6 374 7 117 4 37 8 12 3 7 Name: count, dtype: int64
Y_test.value_counts()
Signal_Strength 5 173 6 161 7 50 4 16 8 5 3 3 Name: count, dtype: int64
2.D Normalise the train and test data with appropriate method.¶
scaled_X_train = pd.DataFrame(scaler.fit_transform(X_train))
scaled_X_test = pd.DataFrame(scaler.fit_transform(X_test))
scaled_X_train.columns = X_train.columns
scaled_X_test.columns = X_test.columns
scaled_X_train.head()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 8 | Parameter 9 | Parameter 10 | Parameter 11 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 1.402648 | -0.150966 | 0.971389 | -0.026726 | -0.318863 | -1.029546 | -0.578278 | 1.825417 | -1.688235 | -0.109826 | -0.861111 |
| 1 | 0.257556 | -0.706894 | 0.662515 | -0.240913 | -0.185844 | 0.336751 | 0.174367 | 0.682066 | 0.856778 | -0.109826 | -1.142107 |
| 2 | -1.288319 | 1.044280 | -0.881856 | 0.044669 | -0.762259 | -0.248805 | -0.578278 | -1.702638 | 1.705115 | -0.970467 | 1.386859 |
| 3 | 0.085792 | -1.374008 | 1.486179 | -0.526496 | 0.102363 | 1.898233 | 1.709761 | 0.083168 | -0.056817 | 0.566391 | 0.075543 |
| 4 | -0.944792 | -1.374008 | 0.044767 | -0.098122 | -1.671221 | 1.995825 | 1.619444 | -3.325107 | -0.317844 | -1.646685 | 1.199528 |
scaled_X_test.head()
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Parameter 5 | Parameter 6 | Parameter 7 | Parameter 8 | Parameter 9 | Parameter 10 | Parameter 11 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2.154619 | -0.710509 | 1.893034 | -0.277008 | -0.376491 | -0.893865 | -0.629735 | 1.125743 | -1.587526 | 0.102980 | 0.752357 |
| 1 | -0.127805 | -1.500608 | 0.783252 | -0.642899 | -0.393721 | 0.489135 | 0.051350 | -1.042803 | 0.425952 | -0.323035 | 1.291386 |
| 2 | 2.622809 | 0.342957 | 1.085920 | 0.251500 | -0.255879 | -1.078265 | -0.866635 | 1.435536 | -1.084157 | -0.482790 | 0.842195 |
| 3 | 0.047766 | 0.026918 | -0.175196 | 0.739353 | -0.238648 | -0.064065 | 1.887319 | 0.506159 | -0.266181 | -0.269783 | -0.954566 |
| 4 | -0.947137 | 1.001374 | -0.931866 | 2.446841 | 0.467795 | -0.709465 | -0.925859 | -0.035978 | 0.237188 | -0.908804 | 0.303167 |
2.E Transform Labels into format acceptable by Neural Network.¶
le = LabelEncoder()
Y_train = le.fit_transform(Y_train)
Y_test = le.fit_transform(Y_test)
3.A Design a Neural Network to train a classifier.¶
Y_train_cat = to_categorical(Y_train)
Y_test_cat = to_categorical(Y_test)
model = Sequential([
Dense(64, activation='relu', input_shape=(scaled_X_train.shape[1],)),
Dropout(0.3), # Regularization
Dense(32, activation='relu'), # Hidden layer
Dropout(0.3), # Regularization
Dense(Y_train_cat.shape[1], activation='softmax')
])
C:\Users\fenuj\anaconda3\envs\tensorflow\Lib\site-packages\keras\src\layers\core\dense.py:87: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead. super().__init__(activity_regularizer=activity_regularizer, **kwargs)
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
3.B Train the classifier using previously designed Architecture.¶
history = model.fit(scaled_X_train, Y_train_cat,
validation_data=(scaled_X_test, Y_test_cat),
epochs=50,
batch_size=32,
verbose=1)
Epoch 1/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 4s 26ms/step - accuracy: 0.1909 - loss: 1.8551 - val_accuracy: 0.4216 - val_loss: 1.4775 Epoch 2/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 30ms/step - accuracy: 0.4193 - loss: 1.4644 - val_accuracy: 0.5564 - val_loss: 1.2556 Epoch 3/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.4770 - loss: 1.3009 - val_accuracy: 0.5539 - val_loss: 1.1530 Epoch 4/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5630 - loss: 1.1310 - val_accuracy: 0.5711 - val_loss: 1.1033 Epoch 5/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5271 - loss: 1.1446 - val_accuracy: 0.5662 - val_loss: 1.0765 Epoch 6/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5620 - loss: 1.1283 - val_accuracy: 0.5662 - val_loss: 1.0610 Epoch 7/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 11ms/step - accuracy: 0.5655 - loss: 1.0931 - val_accuracy: 0.5662 - val_loss: 1.0521 Epoch 8/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5493 - loss: 1.1007 - val_accuracy: 0.5809 - val_loss: 1.0435 Epoch 9/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5671 - loss: 1.1215 - val_accuracy: 0.5809 - val_loss: 1.0364 Epoch 10/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5831 - loss: 1.1094 - val_accuracy: 0.5784 - val_loss: 1.0288 Epoch 11/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.5483 - loss: 1.0530 - val_accuracy: 0.5784 - val_loss: 1.0206 Epoch 12/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step - accuracy: 0.5680 - loss: 1.0745 - val_accuracy: 0.5588 - val_loss: 1.0194 Epoch 13/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6013 - loss: 1.0335 - val_accuracy: 0.5613 - val_loss: 1.0135 Epoch 14/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.5732 - loss: 1.0563 - val_accuracy: 0.5613 - val_loss: 1.0074 Epoch 15/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - accuracy: 0.5825 - loss: 0.9984 - val_accuracy: 0.5711 - val_loss: 1.0057 Epoch 16/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5979 - loss: 1.0238 - val_accuracy: 0.5711 - val_loss: 1.0011 Epoch 17/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5529 - loss: 1.0355 - val_accuracy: 0.5784 - val_loss: 1.0007 Epoch 18/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - accuracy: 0.5766 - loss: 0.9889 - val_accuracy: 0.5637 - val_loss: 0.9974 Epoch 19/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5975 - loss: 1.0109 - val_accuracy: 0.5711 - val_loss: 0.9941 Epoch 20/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.5873 - loss: 1.0357 - val_accuracy: 0.5711 - val_loss: 0.9877 Epoch 21/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.5930 - loss: 0.9961 - val_accuracy: 0.5735 - val_loss: 0.9863 Epoch 22/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6054 - loss: 0.9890 - val_accuracy: 0.5711 - val_loss: 0.9818 Epoch 23/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - accuracy: 0.6079 - loss: 0.9753 - val_accuracy: 0.5833 - val_loss: 0.9802 Epoch 24/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - accuracy: 0.5952 - loss: 0.9868 - val_accuracy: 0.5833 - val_loss: 0.9769 Epoch 25/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - accuracy: 0.5955 - loss: 0.9877 - val_accuracy: 0.5858 - val_loss: 0.9768 Epoch 26/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.6145 - loss: 0.9370 - val_accuracy: 0.5882 - val_loss: 0.9736 Epoch 27/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.5774 - loss: 0.9608 - val_accuracy: 0.5735 - val_loss: 0.9745 Epoch 28/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - accuracy: 0.5863 - loss: 0.9943 - val_accuracy: 0.5760 - val_loss: 0.9735 Epoch 29/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.6055 - loss: 0.9667 - val_accuracy: 0.5760 - val_loss: 0.9752 Epoch 30/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - accuracy: 0.6058 - loss: 0.9218 - val_accuracy: 0.5858 - val_loss: 0.9760 Epoch 31/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.6131 - loss: 0.9651 - val_accuracy: 0.5784 - val_loss: 0.9720 Epoch 32/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.5751 - loss: 1.0031 - val_accuracy: 0.5833 - val_loss: 0.9706 Epoch 33/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6367 - loss: 0.9132 - val_accuracy: 0.5760 - val_loss: 0.9698 Epoch 34/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5997 - loss: 0.9682 - val_accuracy: 0.5907 - val_loss: 0.9670 Epoch 35/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 14ms/step - accuracy: 0.5988 - loss: 0.9791 - val_accuracy: 0.5858 - val_loss: 0.9656 Epoch 36/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6131 - loss: 0.8978 - val_accuracy: 0.5882 - val_loss: 0.9643 Epoch 37/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6208 - loss: 0.9121 - val_accuracy: 0.5809 - val_loss: 0.9634 Epoch 38/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - accuracy: 0.5843 - loss: 0.9728 - val_accuracy: 0.5907 - val_loss: 0.9640 Epoch 39/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6059 - loss: 0.9220 - val_accuracy: 0.5858 - val_loss: 0.9613 Epoch 40/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.5799 - loss: 0.9708 - val_accuracy: 0.5809 - val_loss: 0.9570 Epoch 41/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.6161 - loss: 0.9196 - val_accuracy: 0.5809 - val_loss: 0.9590 Epoch 42/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.6180 - loss: 0.8957 - val_accuracy: 0.5858 - val_loss: 0.9586 Epoch 43/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.6073 - loss: 0.9463 - val_accuracy: 0.5858 - val_loss: 0.9581 Epoch 44/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.5823 - loss: 1.0067 - val_accuracy: 0.5809 - val_loss: 0.9568 Epoch 45/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6489 - loss: 0.8788 - val_accuracy: 0.5833 - val_loss: 0.9578 Epoch 46/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6211 - loss: 0.9298 - val_accuracy: 0.5907 - val_loss: 0.9572 Epoch 47/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5953 - loss: 0.9457 - val_accuracy: 0.5858 - val_loss: 0.9586 Epoch 48/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.6253 - loss: 0.9229 - val_accuracy: 0.5907 - val_loss: 0.9573 Epoch 49/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.6319 - loss: 0.8959 - val_accuracy: 0.5858 - val_loss: 0.9568 Epoch 50/50 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.5847 - loss: 0.9494 - val_accuracy: 0.5858 - val_loss: 0.9559
test_loss, test_accuracy = model.evaluate(scaled_X_test, Y_test_cat)
print(f"Test Loss: {test_loss:.4f}, Test Accuracy: {test_accuracy:.4f}")
13/13 ━━━━━━━━━━━━━━━━━━━━ 0s 9ms/step - accuracy: 0.5277 - loss: 1.0067 Test Loss: 0.9559, Test Accuracy: 0.5858
3.C Plot 2 separate visuals. i. Training Loss and Validation Loss ii. Training Accuracy and Validation Accuracy.¶
# Plot 1: Training Loss and Validation Loss
plt.figure(figsize=(10, 5))
plt.plot(history.history['loss'], label='Training Loss', color='blue')
plt.plot(history.history['val_loss'], label='Validation Loss', color='orange')
plt.title('Training Loss vs. Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.grid(True)
plt.show()
# Plot 2: Training Accuracy and Validation Accuracy
plt.figure(figsize=(10, 5))
plt.plot(history.history['accuracy'], label='Training Accuracy', color='green')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy', color='red')
plt.title('Training Accuracy vs. Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.grid(True)
plt.show()
3.D Design new architecture/update existing architecture in attempt to improve the performance of the model.¶
# Define the revised architecture
model = Sequential([
Dense(128, activation='relu', input_shape=(scaled_X_train.shape[1],), kernel_regularizer=l2(0.001)),
Dropout(0.3),
Dense(64, activation='relu', kernel_regularizer=l2(0.001)),
Dropout(0.3),
Dense(32, activation='relu', kernel_regularizer=l2(0.001)),
Dropout(0.2),
Dense(Y_train_cat.shape[1], activation='softmax') # Output layer
])
# Compile the model
model.compile(optimizer=Adam(learning_rate=0.001),
loss='categorical_crossentropy',
metrics=['accuracy'])
# Learning rate scheduler and early stopping
lr_scheduler = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=5, verbose=1)
early_stopping = EarlyStopping(monitor='val_loss', patience=10, restore_best_weights=True, verbose=1)
# Train the model
history = model.fit(scaled_X_train, Y_train_cat,
validation_data=(scaled_X_test, Y_test_cat),
epochs=100,
batch_size=32,
callbacks=[lr_scheduler, early_stopping],
verbose=1)
C:\Users\fenuj\anaconda3\envs\tensorflow\Lib\site-packages\keras\src\layers\core\dense.py:87: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead. super().__init__(activity_regularizer=activity_regularizer, **kwargs)
Epoch 1/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 4s 33ms/step - accuracy: 0.3200 - loss: 1.9056 - val_accuracy: 0.4804 - val_loss: 1.4824 - learning_rate: 0.0010 Epoch 2/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.4772 - loss: 1.4349 - val_accuracy: 0.5221 - val_loss: 1.2892 - learning_rate: 0.0010 Epoch 3/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.5109 - loss: 1.3415 - val_accuracy: 0.5441 - val_loss: 1.2307 - learning_rate: 0.0010 Epoch 4/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5337 - loss: 1.3376 - val_accuracy: 0.5466 - val_loss: 1.1975 - learning_rate: 0.0010 Epoch 5/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5485 - loss: 1.2889 - val_accuracy: 0.5588 - val_loss: 1.1771 - learning_rate: 0.0010 Epoch 6/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 17ms/step - accuracy: 0.5514 - loss: 1.1984 - val_accuracy: 0.5588 - val_loss: 1.1626 - learning_rate: 0.0010 Epoch 7/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5422 - loss: 1.2099 - val_accuracy: 0.5931 - val_loss: 1.1450 - learning_rate: 0.0010 Epoch 8/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.5585 - loss: 1.1895 - val_accuracy: 0.5882 - val_loss: 1.1310 - learning_rate: 0.0010 Epoch 9/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5749 - loss: 1.1107 - val_accuracy: 0.5809 - val_loss: 1.1248 - learning_rate: 0.0010 Epoch 10/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.5443 - loss: 1.1325 - val_accuracy: 0.5882 - val_loss: 1.1146 - learning_rate: 0.0010 Epoch 11/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5925 - loss: 1.1036 - val_accuracy: 0.5809 - val_loss: 1.1015 - learning_rate: 0.0010 Epoch 12/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step - accuracy: 0.5878 - loss: 1.1355 - val_accuracy: 0.5809 - val_loss: 1.0970 - learning_rate: 0.0010 Epoch 13/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.5907 - loss: 1.0948 - val_accuracy: 0.5882 - val_loss: 1.0882 - learning_rate: 0.0010 Epoch 14/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.6164 - loss: 1.0855 - val_accuracy: 0.5637 - val_loss: 1.0795 - learning_rate: 0.0010 Epoch 15/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.5613 - loss: 1.1032 - val_accuracy: 0.5784 - val_loss: 1.0790 - learning_rate: 0.0010 Epoch 16/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6109 - loss: 1.0756 - val_accuracy: 0.5760 - val_loss: 1.0680 - learning_rate: 0.0010 Epoch 17/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.5996 - loss: 1.0758 - val_accuracy: 0.5858 - val_loss: 1.0642 - learning_rate: 0.0010 Epoch 18/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.5875 - loss: 1.0570 - val_accuracy: 0.5907 - val_loss: 1.0675 - learning_rate: 0.0010 Epoch 19/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5852 - loss: 1.0680 - val_accuracy: 0.5735 - val_loss: 1.0677 - learning_rate: 0.0010 Epoch 20/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.6073 - loss: 1.0493 - val_accuracy: 0.5760 - val_loss: 1.0563 - learning_rate: 0.0010 Epoch 21/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6182 - loss: 1.0396 - val_accuracy: 0.5784 - val_loss: 1.0491 - learning_rate: 0.0010 Epoch 22/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.6029 - loss: 1.0433 - val_accuracy: 0.5809 - val_loss: 1.0503 - learning_rate: 0.0010 Epoch 23/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5996 - loss: 1.0743 - val_accuracy: 0.6005 - val_loss: 1.0444 - learning_rate: 0.0010 Epoch 24/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.5990 - loss: 1.0261 - val_accuracy: 0.6005 - val_loss: 1.0482 - learning_rate: 0.0010 Epoch 25/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6191 - loss: 1.0006 - val_accuracy: 0.5760 - val_loss: 1.0437 - learning_rate: 0.0010 Epoch 26/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.6061 - loss: 1.0049 - val_accuracy: 0.5931 - val_loss: 1.0416 - learning_rate: 0.0010 Epoch 27/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6365 - loss: 0.9969 - val_accuracy: 0.5907 - val_loss: 1.0379 - learning_rate: 0.0010 Epoch 28/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6335 - loss: 1.0059 - val_accuracy: 0.5931 - val_loss: 1.0395 - learning_rate: 0.0010 Epoch 29/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5966 - loss: 1.0291 - val_accuracy: 0.5809 - val_loss: 1.0392 - learning_rate: 0.0010 Epoch 30/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.5883 - loss: 1.0422 - val_accuracy: 0.6005 - val_loss: 1.0383 - learning_rate: 0.0010 Epoch 31/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6022 - loss: 1.0093 - val_accuracy: 0.5931 - val_loss: 1.0355 - learning_rate: 0.0010 Epoch 32/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5940 - loss: 1.0182 - val_accuracy: 0.6054 - val_loss: 1.0293 - learning_rate: 0.0010 Epoch 33/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.6231 - loss: 0.9731 - val_accuracy: 0.6029 - val_loss: 1.0267 - learning_rate: 0.0010 Epoch 34/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.5990 - loss: 1.0067 - val_accuracy: 0.5956 - val_loss: 1.0283 - learning_rate: 0.0010 Epoch 35/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6015 - loss: 0.9491 - val_accuracy: 0.5956 - val_loss: 1.0284 - learning_rate: 0.0010 Epoch 36/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6173 - loss: 0.9694 - val_accuracy: 0.6054 - val_loss: 1.0330 - learning_rate: 0.0010 Epoch 37/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6268 - loss: 0.9495 - val_accuracy: 0.5784 - val_loss: 1.0259 - learning_rate: 0.0010 Epoch 38/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 17ms/step - accuracy: 0.5995 - loss: 1.0134 - val_accuracy: 0.6054 - val_loss: 1.0307 - learning_rate: 0.0010 Epoch 39/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.6215 - loss: 0.9654 - val_accuracy: 0.6029 - val_loss: 1.0209 - learning_rate: 0.0010 Epoch 40/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6071 - loss: 0.9978 - val_accuracy: 0.5956 - val_loss: 1.0253 - learning_rate: 0.0010 Epoch 41/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 13ms/step - accuracy: 0.6008 - loss: 0.9800 - val_accuracy: 0.6078 - val_loss: 1.0268 - learning_rate: 0.0010 Epoch 42/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6099 - loss: 0.9868 - val_accuracy: 0.5907 - val_loss: 1.0257 - learning_rate: 0.0010 Epoch 43/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.5918 - loss: 1.0246 - val_accuracy: 0.6005 - val_loss: 1.0276 - learning_rate: 0.0010 Epoch 44/100 26/30 ━━━━━━━━━━━━━━━━━━━━ 0s 6ms/step - accuracy: 0.6200 - loss: 0.9706 Epoch 44: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6207 - loss: 0.9691 - val_accuracy: 0.5931 - val_loss: 1.0248 - learning_rate: 0.0010 Epoch 45/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6261 - loss: 0.9491 - val_accuracy: 0.5931 - val_loss: 1.0244 - learning_rate: 5.0000e-04 Epoch 46/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.6196 - loss: 0.9426 - val_accuracy: 0.5931 - val_loss: 1.0275 - learning_rate: 5.0000e-04 Epoch 47/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6268 - loss: 0.9424 - val_accuracy: 0.5956 - val_loss: 1.0239 - learning_rate: 5.0000e-04 Epoch 48/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.6514 - loss: 0.9287 - val_accuracy: 0.6005 - val_loss: 1.0252 - learning_rate: 5.0000e-04 Epoch 49/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6233 - loss: 0.9617 - val_accuracy: 0.5882 - val_loss: 1.0187 - learning_rate: 5.0000e-04 Epoch 50/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 16ms/step - accuracy: 0.6125 - loss: 0.9458 - val_accuracy: 0.5980 - val_loss: 1.0213 - learning_rate: 5.0000e-04 Epoch 51/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.6303 - loss: 0.9337 - val_accuracy: 0.5980 - val_loss: 1.0266 - learning_rate: 5.0000e-04 Epoch 52/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6111 - loss: 0.9671 - val_accuracy: 0.5882 - val_loss: 1.0285 - learning_rate: 5.0000e-04 Epoch 53/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6519 - loss: 0.9156 - val_accuracy: 0.5931 - val_loss: 1.0241 - learning_rate: 5.0000e-04 Epoch 54/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.6329 - loss: 0.9448 Epoch 54: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6327 - loss: 0.9451 - val_accuracy: 0.5931 - val_loss: 1.0234 - learning_rate: 5.0000e-04 Epoch 55/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 17ms/step - accuracy: 0.6642 - loss: 0.9230 - val_accuracy: 0.5931 - val_loss: 1.0225 - learning_rate: 2.5000e-04 Epoch 56/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6353 - loss: 0.9473 - val_accuracy: 0.5956 - val_loss: 1.0219 - learning_rate: 2.5000e-04 Epoch 57/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.6114 - loss: 0.9519 - val_accuracy: 0.5980 - val_loss: 1.0253 - learning_rate: 2.5000e-04 Epoch 58/100 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.6341 - loss: 0.9274 - val_accuracy: 0.5956 - val_loss: 1.0238 - learning_rate: 2.5000e-04 Epoch 59/100 21/30 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.6818 - loss: 0.9017 Epoch 59: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 30/30 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.6751 - loss: 0.9103 - val_accuracy: 0.5931 - val_loss: 1.0226 - learning_rate: 2.5000e-04 Epoch 59: early stopping Restoring model weights from the end of the best epoch: 49.
3.E Plot visuals as in Q3.C and share insights about difference observed in both the models.¶
# Plot 1: Training Loss vs. Validation Loss
plt.figure(figsize=(10, 5))
plt.plot(history.history['loss'], label='Training Loss', color='blue')
plt.plot(history.history['val_loss'], label='Validation Loss', color='orange')
plt.title('Training Loss vs. Validation Loss (Updated Model)')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.grid(True)
plt.show()
# Plot 2: Training Accuracy vs. Validation Accuracy
plt.figure(figsize=(10, 5))
plt.plot(history.history['accuracy'], label='Training Accuracy', color='green')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy', color='red')
plt.title('Training Accuracy vs. Validation Accuracy (Updated Model)')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.grid(True)
plt.show()
Insights¶
Training vs. Validation Loss:¶
Initial Model:¶
- The training loss decreased rapidly, but the validation loss either plateaued early or increased, indicating potential overfitting.
Updated Model:¶
- A smoother convergence between training and validation loss suggests that regularization techniques (Dropout, L2) helped mitigate overfitting.
Training vs. Validation Accuracy:¶
Initial Model:¶
- A significant gap between training and validation accuracy was observed, indicating poor generalization.
Updated Model:¶
- The gap between training and validation accuracy is smaller, showing better generalization to unseen data.
Key Differences Observed:¶
Overfitting Reduction:¶
Regularization (Dropout, L2) likely reduced overfitting, evident in closer training and validation performance.
Smoother Convergence:¶
With the learning rate scheduler, loss and accuracy curves should show smoother convergence.
Improved Generalization:¶
The improved model should exhibit higher validation accuracy and lower validation loss compared to the initial model.
Part B¶
1.A Read the .h5 file and assign to a variable.¶
import h5py
print(h5py.__version__)
3.12.1
ah5 = h5py.File("Autonomous_Vehicles_SVHN_single_grey1 (1).h5")
1.B Print all the keys from the .h5 file.¶
list(ah5.keys())
['X_test', 'X_train', 'X_val', 'y_test', 'y_train', 'y_val']
1.C Split the data into X_train, X_test, Y_train, Y_test¶
X_train_img, X_test_img, Y_train_img, Y_test_img = ah5['X_train'], ah5['X_test'], ah5['y_train'], ah5['y_test']
print(type(X_train_img), type(X_test_img), type(Y_train_img), type(Y_test_img))
<class 'h5py._hl.dataset.Dataset'> <class 'h5py._hl.dataset.Dataset'> <class 'h5py._hl.dataset.Dataset'> <class 'h5py._hl.dataset.Dataset'>
X_train_img, X_test_img, Y_train_img, Y_test_img = X_train_img[:], X_test_img[:], Y_train_img[:], Y_test_img[:]
print(type(X_train_img), type(X_test_img), type(Y_train_img), type(Y_test_img))
<class 'numpy.ndarray'> <class 'numpy.ndarray'> <class 'numpy.ndarray'> <class 'numpy.ndarray'>
2.A Print shape of all the 4 data split into x, y, train, test to verify if x & y is in sync.¶
print('X train:', X_train_img.shape, '| X test:', X_test_img.shape, '| Y train:', Y_train_img.shape, '| Y test:', Y_test_img.shape)
X train: (42000, 32, 32) | X test: (18000, 32, 32) | Y train: (42000,) | Y test: (18000,)
2.B Visualise first 10 images in train data and print its corresponding labels.¶
for i in np.arange(10):
plt.imshow(X_train_img[i])
plt.show()
print('\nindex: ', i, '| Label: ', Y_train_img[i])
print('_' * 55, '\n')
index: 0 | Label: 2 _______________________________________________________
index: 1 | Label: 6 _______________________________________________________
index: 2 | Label: 7 _______________________________________________________
index: 3 | Label: 4 _______________________________________________________
index: 4 | Label: 4 _______________________________________________________
index: 5 | Label: 0 _______________________________________________________
index: 6 | Label: 3 _______________________________________________________
index: 7 | Label: 0 _______________________________________________________
index: 8 | Label: 7 _______________________________________________________
index: 9 | Label: 3 _______________________________________________________
2.C Reshape all the images with appropriate shape update the data in same variable.¶
X_train_img = X_train_img.reshape((X_train_img.shape[0], -1))
X_test_img = X_test_img.reshape((X_test_img.shape[0], -1))
print('X train:', X_train_img.shape, '| X test:', X_test_img.shape, '| Y train:', Y_train_img.shape, '| Y test:', Y_test_img.shape)
X train: (42000, 1024) | X test: (18000, 1024) | Y train: (42000,) | Y test: (18000,)
2.D Normalise the images i.e. Normalise the pixel values.¶
scaled_X_train_img = pd.DataFrame(scaler.fit_transform(X_train_img))
scaled_X_test_img = pd.DataFrame(scaler.fit_transform(X_test_img))
2.E Transform Labels into format acceptable by Neural Network¶
Y_train_img = to_categorical(Y_train_img)
Y_test_img = to_categorical(Y_test_img)
2.F Print total Number of classes in the Dataset.¶
print(Y_train_img.shape)
(42000, 10)
3.A Design a Neural Network to train a classifier.¶
model_img_1 = Sequential()
model_img_1.add(Dense(50, input_shape = (1024, )))
model_img_1.add(Activation('sigmoid'))
model_img_1.add(Dense(50))
model_img_1.add(Activation('sigmoid'))
model_img_1.add(Dense(50))
model_img_1.add(Activation('sigmoid'))
model_img_1.add(Dense(50))
model_img_1.add(Activation('sigmoid'))
model_img_1.add(Dense(10))
model_img_1.add(Activation('softmax'))
sgd = optimizers.SGD(learning_rate = 0.01)
model_img_1.compile(optimizer = sgd, loss = 'categorical_crossentropy', metrics = ['accuracy'])
3.B Train the classifier using previously designed Architecture (Use best suitable parameters).¶
from tensorflow.keras.layers import ReLU
model_img_1.add(Dense(50))
model_img_1.add(ReLU())
from tensorflow.keras.layers import BatchNormalization
model_img_1.add(Dense(50))
model_img_1.add(BatchNormalization())
model_img_1.add(ReLU())
from tensorflow.keras.optimizers import Adam
adam = Adam(learning_rate=0.001)
model_img_1.compile(optimizer=adam, loss='categorical_crossentropy', metrics=['accuracy'])
from tensorflow.keras.layers import Dropout
model_img_1.add(Dropout(0.5))
model_img_1.add(Dense(10)) # Final layer should have 10 neurons
model_img_1.add(Activation('softmax')) # Use softmax for multi-class classification
model.add(Activation('relu'))
# Using ReLU activations and Adam optimizer for better results
def create_model():
model = Sequential()
model.add(Dense(50, input_shape=(1024,)))
model.add(Activation('relu')) # Use ReLU instead of sigmoid
model.add(Dense(50))
model.add(Activation('relu')) # ReLU for hidden layers
model.add(Dense(50))
model.add(Activation('relu'))
model.add(Dense(50))
model.add(Activation('relu'))
model.add(Dense(10)) # Output layer
model.add(Activation('softmax')) # Softmax for classification
# Use Adam optimizer
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
return model
# Train the model
model_img_1 = create_model()
history_img_1 = model_img_1.fit(scaled_X_train_img, Y_train_img, batch_size=500, epochs=50, verbose=1)
Epoch 1/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 3s 8ms/step - accuracy: 0.1065 - loss: 2.3079 Epoch 2/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.2202 - loss: 2.2079 Epoch 3/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 7ms/step - accuracy: 0.3750 - loss: 1.7700 Epoch 4/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.4738 - loss: 1.5425 Epoch 5/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.5250 - loss: 1.4200 Epoch 6/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.5627 - loss: 1.3308 Epoch 7/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.5854 - loss: 1.2715 Epoch 8/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.6030 - loss: 1.2271 Epoch 9/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 2s 11ms/step - accuracy: 0.6180 - loss: 1.1828 Epoch 10/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 7ms/step - accuracy: 0.6273 - loss: 1.1666 Epoch 11/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.6333 - loss: 1.1563 Epoch 12/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.6415 - loss: 1.1213 Epoch 13/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 7ms/step - accuracy: 0.6501 - loss: 1.0982 Epoch 14/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.6548 - loss: 1.0856 Epoch 15/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.6647 - loss: 1.0668 Epoch 16/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.6675 - loss: 1.0574 Epoch 17/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.6768 - loss: 1.0321 Epoch 18/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 2s 12ms/step - accuracy: 0.6744 - loss: 1.0277 Epoch 19/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 11ms/step - accuracy: 0.6834 - loss: 1.0096 Epoch 20/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.6902 - loss: 0.9888 Epoch 21/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 11ms/step - accuracy: 0.6974 - loss: 0.9787 Epoch 22/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 11ms/step - accuracy: 0.7037 - loss: 0.9474 Epoch 23/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7003 - loss: 0.9600 Epoch 24/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.6994 - loss: 0.9498 Epoch 25/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7102 - loss: 0.9300 Epoch 26/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.7141 - loss: 0.9214 Epoch 27/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.7193 - loss: 0.9043 Epoch 28/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7158 - loss: 0.9099 Epoch 29/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.7267 - loss: 0.8924 Epoch 30/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7216 - loss: 0.8891 Epoch 31/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.7318 - loss: 0.8677 Epoch 32/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.7356 - loss: 0.8629 Epoch 33/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.7373 - loss: 0.8509 Epoch 34/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.7368 - loss: 0.8490 Epoch 35/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 2s 10ms/step - accuracy: 0.7365 - loss: 0.8480 Epoch 36/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.7392 - loss: 0.8421 Epoch 37/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.7467 - loss: 0.8198 Epoch 38/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7460 - loss: 0.8113 Epoch 39/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7480 - loss: 0.8093 Epoch 40/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7500 - loss: 0.8046 Epoch 41/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7472 - loss: 0.8156 Epoch 42/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7529 - loss: 0.7996 Epoch 43/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7563 - loss: 0.7887 Epoch 44/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 8ms/step - accuracy: 0.7543 - loss: 0.7911 Epoch 45/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 11ms/step - accuracy: 0.7558 - loss: 0.7905 Epoch 46/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.7604 - loss: 0.7730 Epoch 47/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.7586 - loss: 0.7782 Epoch 48/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.7591 - loss: 0.7662 Epoch 49/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 10ms/step - accuracy: 0.7691 - loss: 0.7516 Epoch 50/50 84/84 ━━━━━━━━━━━━━━━━━━━━ 1s 11ms/step - accuracy: 0.7642 - loss: 0.7579
def improved_model():
model = Sequential()
model.add(Dense(128, input_shape=(1024,), activation='relu', kernel_initializer='he_normal'))
model.add(Dense(64, activation='relu', kernel_initializer='he_normal'))
model.add(Dense(10, activation='softmax', kernel_initializer='he_normal'))
model.compile(optimizer=Adam(learning_rate=0.001), loss='categorical_crossentropy', metrics=['accuracy'])
return model
scaled_X_train_img = X_train_img / 255.0
scaled_X_test_img = X_test_img / 255.0
from tensorflow.keras.callbacks import EarlyStopping
early_stopping = EarlyStopping(monitor='val_loss', patience=5, restore_best_weights=True)
model_img_2 = improved_model()
history_img_2 = model_img_2.fit(scaled_X_train_img, Y_train_img, batch_size=1000, epochs = 20, verbose = 1)
Epoch 1/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 3s 20ms/step - accuracy: 0.0978 - loss: 2.3601 Epoch 2/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 19ms/step - accuracy: 0.1330 - loss: 2.2900 Epoch 3/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 17ms/step - accuracy: 0.1894 - loss: 2.2507 Epoch 4/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 19ms/step - accuracy: 0.2465 - loss: 2.1613 Epoch 5/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - accuracy: 0.3360 - loss: 1.9949 Epoch 6/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - accuracy: 0.4278 - loss: 1.8019 Epoch 7/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - accuracy: 0.4791 - loss: 1.6381 Epoch 8/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 19ms/step - accuracy: 0.5361 - loss: 1.5045 Epoch 9/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 17ms/step - accuracy: 0.5603 - loss: 1.4171 Epoch 10/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - accuracy: 0.5919 - loss: 1.3441 Epoch 11/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 18ms/step - accuracy: 0.6032 - loss: 1.2970 Epoch 12/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 19ms/step - accuracy: 0.6262 - loss: 1.2400 Epoch 13/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 18ms/step - accuracy: 0.6365 - loss: 1.2107 Epoch 14/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 19ms/step - accuracy: 0.6464 - loss: 1.1759 Epoch 15/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - accuracy: 0.6598 - loss: 1.1340 Epoch 16/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 19ms/step - accuracy: 0.6577 - loss: 1.1340 Epoch 17/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 19ms/step - accuracy: 0.6670 - loss: 1.1039 Epoch 18/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 21ms/step - accuracy: 0.6787 - loss: 1.0782 Epoch 19/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - accuracy: 0.6800 - loss: 1.0639 Epoch 20/20 42/42 ━━━━━━━━━━━━━━━━━━━━ 1s 20ms/step - accuracy: 0.6879 - loss: 1.0402
3.C Evaluate performance of the model with appropriate metrics.¶
results_img_1 = model_img_1.evaluate(scaled_X_test_img, Y_test_img)
563/563 ━━━━━━━━━━━━━━━━━━━━ 2s 3ms/step - accuracy: 0.7462 - loss: 0.8272
3.D Plot the training loss, validation loss vs number of epochs and training accuracy, validation accuracy vs number of epochs plot and write your observations on the same.¶
history_img_1 = model_img_1.fit(scaled_X_train_img, Y_train_img,
batch_size=500, epochs=10, verbose=1,
validation_split=0.2)
Epoch 1/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 2s 12ms/step - accuracy: 0.7632 - loss: 0.7690 - val_accuracy: 0.7533 - val_loss: 0.7819 Epoch 2/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.7676 - loss: 0.7598 - val_accuracy: 0.7590 - val_loss: 0.7791 Epoch 3/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.7708 - loss: 0.7427 - val_accuracy: 0.7695 - val_loss: 0.7475 Epoch 4/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 13ms/step - accuracy: 0.7688 - loss: 0.7405 - val_accuracy: 0.7725 - val_loss: 0.7393 Epoch 5/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.7729 - loss: 0.7402 - val_accuracy: 0.7631 - val_loss: 0.7626 Epoch 6/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 14ms/step - accuracy: 0.7691 - loss: 0.7366 - val_accuracy: 0.7670 - val_loss: 0.7542 Epoch 7/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 11ms/step - accuracy: 0.7766 - loss: 0.7205 - val_accuracy: 0.7567 - val_loss: 0.7934 Epoch 8/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.7730 - loss: 0.7378 - val_accuracy: 0.7624 - val_loss: 0.7628 Epoch 9/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 15ms/step - accuracy: 0.7721 - loss: 0.7291 - val_accuracy: 0.7726 - val_loss: 0.7442 Epoch 10/10 68/68 ━━━━━━━━━━━━━━━━━━━━ 1s 12ms/step - accuracy: 0.7767 - loss: 0.7240 - val_accuracy: 0.7700 - val_loss: 0.7567
# Extract values from the history object
train_loss = history_img_1.history['loss']
val_loss = history_img_1.history['val_loss']
train_accuracy = history_img_1.history['accuracy']
val_accuracy = history_img_1.history['val_accuracy']
# Plot Training and Validation Loss
plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.plot(train_loss, label='Training Loss')
plt.plot(val_loss, label='Validation Loss')
plt.title('Training vs Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
# Plot Training and Validation Accuracy
plt.subplot(1, 2, 2)
plt.plot(train_accuracy, label='Training Accuracy')
plt.plot(val_accuracy, label='Validation Accuracy')
plt.title('Training vs Validation Accuracy')
plt.xlabel('Epochs')
plt.ylabel('Accuracy')
plt.legend()
plt.tight_layout()
plt.show()
Observations:¶
Training Loss:¶
The training loss should generally decrease over time as the model learns. If it doesn't decrease or plateaus at a high value, it may indicate an issue with the model architecture, data, or optimization parameters.
Validation Loss:¶
The validation loss typically tracks the model's generalization ability. If it starts increasing while the training loss decreases, it may suggest overfitting.
Training Accuracy:¶
This metric should improve over time as the model learns the patterns in the data. A consistent increase is a good sign of model improvement.
Validation Accuracy:¶
A significant gap between training and validation accuracy indicates overfitting, while similar trends in both curves show that the model generalizes well.
Insights¶
Training vs Validation Loss¶
Overall Trend:¶
The training loss and validation loss both show a decreasing trend, which is expected during training. The model is learning and improving its ability to predict the target variable.
Validation Loss Higher:¶
The validation loss is consistently higher than the training loss. This is a common observation and suggests that the model is overfitting to the training data. It is performing better on the data it has seen during training but struggles to generalize to unseen data.
Training vs Validation Accuracy¶
Overall Trend:¶
Both training and validation accuracy are increasing, indicating that the model is becoming more accurate in its predictions.
Validation Accuracy Lower:¶
Similar to the loss, the validation accuracy is lower than the training accuracy. This again points to overfitting. The model is memorizing patterns in the training data rather than learning generalizable features.
The consistent gap between training and validation metrics strongly indicates overfitting.
Recommentations¶
Based on the observed overfitting, I recommend implementing regularization techniques and monitoring validation performance closely during training. Early stopping can also be a valuable strategy to prevent further overfitting.